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Introduction 


Hate crime,” a relatively new term in American jurisprudence, refers to crimes 
, y 


specifically motivated by an animus against a particular characteristic as enumerated by statute. 
This report focuses on a matter related to hate crimes: the use of electronic communications 
media to advocate or encourage the commission of hate crimes. 


Because electronic communications by their nature involve expression that often 
combines images, sound, and text, government s response to these concerns must be tempered 


against other cherished rights, particularly those guaranteed by the First Amendment. In 1992, 
Congress directed the National Telecommunications and Information Administration (NTIA) to 
undertake an examination of the use of telecommunications, including broadcast radio and 
television, cable television, public access television, and computer bulletin boards, to advocate or 
encourage violent acts, and the commission of crimes of hate against designated persons or 
groups. 


In 1993, responding to a congressional mandate, NTIA published 7he Role of 
Telecommunications in Hate Crimes.' (1993 Report). This report, written under the direction of 
Clinton Administration Commerce Secretary Ronald Brown, surveyed the nature and frequency 
of instances in which telecommunication has been used to advocate or encourage the 
commission of hate crimes. 


The report concluded that no rigorous data linked the problem of hate crimes to use of 
telecommunications services. Rather, while troubling, the links between hate crimes and 
telecommunications use were found to be for the most part anecdotal. NTIA, in that report, 
endorsed the belief that the best remedy for hateful communications is not government 
restrictions, but more speech to disseminate views that challenge notions of hate and bigotry. 
Following the tradition in English and American political theory of viewing free expression as 
the keystone of democratic society, NTIA endorsed the use of discussion, reason, and a wide and 
diverse public square to counter hateful word and deed and build a stronger, more just, America.” 


The world has undergone significant changes since the publication of the 1993 Report, 
with perhaps one of the most dramatic and impactful developments being the explosion of 
information communications technologies and their varied uses, including the emergence of the 
modern Internet, social media, texting, and smart phones. These technologies and their uses 
have paid incredible economic and social dividends, from enabling remote learning and work 
during the COVID-19 pandemic, to empowering individuals to find or found communities and 
connections that would otherwise be impossible. Congress reasoned that these changes warrant a 
revisiting of the 1993 Report. 





1 The Role of Telecommunications in Hate Crimes, NTIA Report to Congress (Dec. 1993) available at 
https://www.ntia.doc.gov/legacy/reports/1993/TelecomHateCrimes1993.pdf. 
2 Id. at 46. 





In 2019, in the accompanying report (H. Rept. 116-101) to the Commerce, Justice, 
Science, and Related Agencies Appropriations bill,? Congress instructed NTIA to provide an 


update to its prior report,” which should analyze the use of new forms of electronic media in 


advocating and encouraging the commission of hate crimes and include any recommendations to 
address such use of telecommunications, consistent with the First Amendment.” * This report has 
been drafted in response to this request. 


We arrive at conclusions similar to those in the 1993 Report. We find no evidence that 
electronic communications, including the internet, cause hate crimes. Further, we found no 
evidence that hate criminals use electronic communications, including the Internet, more than 
any other type of criminals or other types of communication. We caution that efforts to control or 
monitor online speech, even for the worthy goal of reducing crime, present serious First 


Amendment concerns and runs counter to our nation s dedication to free expression. To quote 


President Barack Obama, The strongest weapon against hateful speech is not repression; it is 
more speech.”> 


At the same time, NTIA recognizes that certain criminal groups, many of which are 
terrorist organizations that forward hateful ideologies, use social media and other electronic 
communications. They, like all criminals, are further empowered through speech that solicits, 
conspires, and aids and abets hate crimes. We urge further vigilance against individuals who use 
electronic media to solicit, conspire, or aid and abet hate crimes and recommend policies to 
strengthen our efforts to improve our abilities to counter such acts. 


To that end, we urge reconsideration of section 230,° either by Congress or the Federal 
Communications Commission. This provision provides that [n]o provider or user of an 


interactive computer service shall be treated as the publisher or speaker of any information 
provided by another information content provider.” 5 U.S.C. § 230(c)(1) It thus provides broad 
immunity for platforms that publish unlawful speech of third parties. Although hate speech is 
legal, it is unlawful to solicit a hate crime, such as an assault against an individual because of a 
protected characteristic, or to conspire to commit hate crimes. But, platforms that re-publish 
criminal speech that solicits, conspires, or aids and abets hate crimes face no criminal liability. 
Serious enforcement of hate crimes requires reform of section 230.’ 


Congress instructed NTIA to analyze the use of new forms of electronic media in 
advocating and encouraging the commission of hate crimes.” The report, therefore, sets forth 


3H. Rept. 116-101, Commerce, Justice, Science, and Related Agencies Appropriations Bill (2020) 
available at https://www.congress.gov/congressional-report/1 16th-congress/house-report/101/1. 

4 See id. 

> Remarks by the President to the UN General Assembly, United Nations Headquarters, New York, New 
York (Sept. 25, 2012). 

647 U.S.C. Section 230. 

7 An important guide in advancing useful answers to this difficult legislative challenge can be found in the 
Department of Justice’s recent proposals for revising section 230. See https://www.justice.gov/ag/department- 
justice-s-review-section-230-communications-decency-act-1996 





(1) a discussion of some hate crime” statutes and data concerning the prevalence of hate crimes; 


(2) an analysis of the Supreme Court s First Amendment precedent regarding threatening 


communications and its relationship to hate crimes committed online; (3) a survey of the 
literature describing instances of advocating and encouraging hate crimes; and (4) an analysis of 
any causal relationship between the advocacy and encouragement of hate crimes and the 
commission of such hate crimes. Finally, the report will examine efforts by dominant social 
media platforms to ban advocacy and encouragement of hate crimes by their users, which is 
often done pursuant to platforms’ so-called hate speech” policies. The report recommends 


continued vigilance in this area, combined with a recognition that excessive desires to stamp out 
hate may lead to diminished freedoms and impoverished public discourse. 


Hate Crimes in the United States 


Prosecuting Federal Hate Crimes'® lists the four main federal hate-crime statutes: 


e Title 18, U.S.C. § 249, the Hate Crime Prevention Act, which prohibits willfully causing 
bodily injury—or attempting to cause bodily injury with a dangerous weapon—if the 
defendant is motivated by the actual or perceived race, color, religion, national origin, 
gender, gender identity, sexual orientation, or disability of any person. However, it does 
not go as far as to prohibit vandalism or mere threats. 


e Title 42, U.S.C. § 3631, which prohibits using or threatening to use force to willfully 
injure, intimidate, or interfere with an individual s housing rights as set forth in the 


statute, or attempt to do so, an individual s housing rights as set forth in the statute 


because of the victim s race, color, religion, sex, handicap, familial status, or national 
origin. 

e Title 18, U.S.C. § 245(b)(2), which prohibits using force or the threat of force to willfully 
injure, intimidate, or interfere with a person, or attempt to do so, because of that person s 


race, color, religion, or national origin; and because that person was enjoying one of the 
rights protected by the statute, such as the right to attend a public school or college, the 
right to employment, the right to public accommodations, or the right to enjoy a benefit, 
service, program, or facility provided by a state or local government. 

e Title 18 U.S.C. § 247, which prohibits intentionally defacing, damaging, or destroying 
religious real property and which also prohibits using force or threats of force to obstruct 
someone in the free exercise of religion or to attempt to do so. 


In addition to these statutes, prosecutors may also charge other federal statutes in hate-crime 
prosecutions, including general criminal statutes, as well as the federal civil rights conspiracy 





8 Barbara Kay Bosserman; Angela M. Miller, “Prosecuting Federal Hate Crimes,” UNITED STATES 
ATTORNEYS’ BULLETIN 66, no. | at 191-236. (Jan. 2018), 
https://heinonline.org/HOL/Page?handle=hein.journals/usab66&id=1 93 &collection=journals&index=. 





statute, 18 U.S.C. § 241, which prohibits two or more individuals from infringing on another s 
free exercise or enjoyment of any right or privilege secured by the constitution or federal law.’ 
Furthermore, as explained by the United States Sentencing Commission s Sentencing Guidelines 


Manual provides this three-level enhancement when a finder of fact or the court at sentencing (in 
the case of a nolo contendere or guilty plea) finds that the offender was motivated by bias in 
choosing his victim. In addition, a general conspiracy statute may be used to federally prosecute 
hate crimes.!° Title 18, Section 241, titled Conspiracy Against Rights, prohibits two or more 


individuals from infringing on another s free exercise or enjoyment of any constitutional 


privilege or right and can punish conspirators with imprisonment for life or even the death 
penalty (if the acts result in death of the victim or if they include kidnapping/attempt to kidnap, 
aggravated sexual abuse/attempt to commit aggravated sexual abuse, or attempt to kill). '! 


In addition to the above federal laws, each state has its own penal code which results in 
varied hate crime laws across the United States. Alison Smith from The Congressional Research 
Service issued a report, State Statutes Governing Hate Crimes,'? which describes the differences 


in each state s approach. The report divides hate-crime laws into four categories: crimes and 
penalty enhancements, institutional vandalism, data collection, and law enforcement training. 


While almost all states and many localities have some kind of penalty enhancement for 
crimes motivated by bias, they vary considerably as to what motivations and biases they cover. 
For example, California defines a hate crime to be a criminal act committed in whole or in part 


because of one of the following actual or perceived characters of the victim: Race, color, 


religion, national origin, gender, sexual orientation, gender identity, disability. ethnicity, 
association with a person or group with one or more of these actual or perceived 


characteristics.” '? Michigan requires local police to provide information about crimes motivated 
by prejudice or bias based on race, religion, gender, sexual orientation, ethnic origin.” '* The 
City of Seattle has a malicious harassment” statute that prohibits a defendant from causing 
injury to another, threatening another, or destroying another s property because of his or her 


perception of another person s in turns looks to an even wider array of biases, beyond racial or 


religious animus, and includes crimes motivated by homelessness, marital status, political 
ideology or party, age, and parental status.”!° 





° 18 U.S.C. § 241. 

10 See Barbara Kay Bosserman; Angela M. Miller, “Prosecuting Federal Hate Crimes,” UNITED STATES 
ATTORNEYS’ BULLETIN 66, no. | at 192 (January 2018), 
https://heinonline.org/HOL/Page?handle=hein.journals/usab66&id=193 &collection=journals&index=. 
™ 18 U.S.C. § 241. 

? Alison Smith, “State Statutes Governing Hate Crimes,” U.S. Congressional Research Service, RL 
33099 (Sept. 28, 2010) available at https://fas.org/sgp/crs/misc/RL33099.pdf. 

*3 Cal. Penal Code Tit. 11.6 §§ 422.55 — 422.57. 

™ Mich. Comp. Laws § 28.257A. 

'S Seattle City Code, §12A.06.115 - Malicious harassment, available at 








In Wisconsin v. Mitchell,'® the Supreme Court ruled that enhanced sentencing based on 
biased motivation does not violate the First Amendment. While the Supreme Court did uphold 
statutes enhancing punishments for bias crimes against constitutional challenges in Mitchell, 


many commentators have concluded that hate crimes present unique jurisprudential challenges. '” 


They argue that the justifications for hate crimes fail to provide the necessary theoretical 


218 


legitimacy for this politically popular form of criminal legislation,” ’° as their foundations are 


empirically, morally, or conceptually suspect.”!” 


Beyond theoretical concerns, hate crimes can present evidentiary challenges as well. 
These concerns relate to juries’ ability to establish beyond a reasonable doubt whether a 
defendant committed a particular crime with the covered bias motivation that the hate crime 
statute prohibits. Because most crimes require juries to infer defendants’ mental states from 


observable facts, general intent crimes that prohibit simple acts typically do not overtax juries 


abilities. As Justice Oliver Wendell Holmes famously remarked, Even a dog distinguishes 
between being stumbled over and being kicked.” 


In contrast, hate crimes “have as elements a particular reason for acting, a need or desire 
that causes the person to act.” ?! These crimes require the finding of both the required mental 
state and the required motivation. As criminal researchers James B. Jacobs and Kimberly A. 
Potter point out, mixed motives complicate mental state issues in hate crimes.” 


These conceptual problems have led to challenges for researchers to determine the 
prevalence of hate crimes as well as to determine their causal drivers; these conceptual problems 


also render government policymakers decisions difficult. Lamenting the imprecision in 


terminology and lack of unity in practice, researchers comment that while [mJost agree that a 


hate crime is a crime committed due to bias; the controversy is usually centered on the reasons 
causing the bias and the question of whether other elements are required. However, this basic 
premise may not be enough to bridge between theory and practice. The term bias is, in itself, 


https://library.municode.com/wa/seattle/codes/municipal_code?nodeId=TIT12ACRCO_ SUBTITLE ICR 
CO_CH12A.06OFAGPE 12A.06.115MAHA. 

16 Wisconsin v. Mitchell, 508 U.S. 476 (1993). 

17 James B. Jacobs & Kimberly Potter, “Hate Crimes: Criminal Law And Identity Politics,’ OXFORD 
UNIVERSITY PRESS (1998); Steven G. Gey, “What if Wisconsin v. Mitchell Had Involved Martin Luther 
King Jr.? The Constitutional Flaws of Hate Crime Enhancement Statutes,” 65 GEO. WASH. L. REV. 1014, 
1070 (1997) (explaining that “the Court blinked” in Mitchell, despite its good record for using “liberal 
principles on behalf of an illiberal and reviled defendant.’’). 

18 Heidi M. Hurd & Michael S. Moore, “Punishing Hatred and Prejudice,” 56 STAN. L. REV. at 1081, 
1146 (2004). 

dd. 

20 Oliver Wendell Holmes, Jr., “The Common Law,” at 3 (1881). 

21 Paul H. Robinson, “Hate Crimes: Crimes of Motive, Character, or Group Terror?,” ANN. SURV. AM. L. 
at 605, 606 (1993). 

22 «Hate Crimes: A Critical Perspective,” 22 CRIME & JUST. 1, at 5 (1997). 














highly problematic. . . . [r]esearch in all disciplines lacks a valid methodological basis, due to the 
absence of official data gathering based on common criteria . . . creating incoherent and 
discriminative enforcement.” 


These theoretical concerns affect how we analyze, as Congress tasked us, the use of new 


forms of electronic media in advocating and encouraging the commission of hate crimes.” To 
advocate or encourage hate crimes, one must not simply express disdain or even hatred for a 
particular group. In other words, merely expressing hatred for a particular group cannot be said 


to advocat[e] or encourage[e]” hate crimes. Therefore, so-called hate speech,” a term that has 


no consistent or well-accepted definition, cannot constitute advocacy or encouragement of hate 
crimes without explicitly calling for particular acts to be performed with specific motivations. 
Rather, to advocate for or encourage hate crimes requires advocacy of particular criminal acts as 
well as advocacy or encouragement that the criminal act of each of such crimes have a particular 
motivation. 


Hate Crime Data 


There are two main sources of federal data related to hate crimes in the United States. 
Neither source, due in part to collection limitations, provide evidence that electronic media play 
a significant role in encouraging or advocating hate crimes. 


First, the Justice Department s Uniform Crime Reporting (UCR) program includes data 


collection on reported incidents of possible hate crimes (among other types of crimes) reported 
by thousands of law enforcement agencies around the country. This is an updated version of the 
same data collection NTIA noted in its 1993 Report on the role of telecommunications in hate 
crimes. Since 2016, participating enforcement agencies have had the option of specifying 


cyberspace” as the location of a hate crime,”* which is defined as a virtual or internet-based 


network of two or more computers in separate locations which communicate either through 
wireless or wire connections.”*> In 2019, 36 out of the 7,314 hate crime incidents submitted to 
the UCR program reportedly occurred in cyberspace.”° 


Table 177 


23 Hitman, Gadi & Harel, Dror, “Hate Crimes—Methodological, Theoretical & Empirical Difficulties—A 
Pragmatic & Legal Overview,” JOURNAL OF CULTURAL AND RELIGIOUS STUDIES, at 4 (2016). 

49018 Hate Crime Statistics, Criminal Justice Information Services Division, available at 
https://ucr.fbi.gov/hate-crime/2018/resource-pages/methodology. 

5 2019.2 National Incident-Based Reporting System User Manual at 87, available at 
https://www.fbi.gov/file-repository/ucr/ucr-2019-1-nibrs-user-manual.pdf/view. 

6 2018 Hate Crime Statistics, Criminal Justice Information Services Division, Table 10, available at 
https://ucr.fbi.gov/hate-crime/2018/tables/table-10.xls. 

7 FBI UCR Program’s Hate Crime Statistics Publication, available at https://ucr.fbi.gov/hate-crime. 
https://ucr.fbi.gow/hate-crime. 





















































Number of law Number of law enforcement agencies Number of hate 
Year enforcement agencies participated in the Hate Crime crime incidents 

reporting data Statistics Program reported?® 
2008 2,145 13,690 7,783 
2009 2,034 14,422 6,604 
2010 1,949 14,977 6,628 
2011 1,944 14,575 6,222 
2012 1,730 13,022 5,796 
2013 1,826 15,016 5,928 
2014 1,666 15,494 5,479 
2015 1,742 14,997 5,850 
2016 1,776 15,254 6,121 
2017 2,040 16,149 7,175 
2018 2,026 16,039 7,120 
2019 2,172 15,588 7,314 














Second, the other major national source of government hate crimes statistics is the 
National Crime Victimization Survey (NCVS). The NCVS is commissioned by the Bureau of 
Justice Statistics (BJS) and administered by the U.S. Census Bureau. In contrast to the UCR 
program, which is based on incident reports summarized by participating law enforcement 
agencies and submitted to the Justice Department, the NCVS is an annual survey of 
approximately 95,000 households around the country. Furthermore, respondents are asked about 
all incidents of criminal victimization they may have experienced, not just those ultimately 
reported to police. Among other topics, respondents are asked whether they believe reported 
incidents were hate crimes or otherwise motivated by prejudice or bigotry, and if so, they are 
asked for various related details. However, those details do not include any connection between 
the incident and telecommunications systems.”? From 2013 to 2017, the NCVS reported an 
average of 204,600 hate crime victimizations, of which approximately half were reported to the 
police.*° 





*8 These are reported crime. Data is not collected on convictions. 

9 National Crime Victimization Survey, NCVS-2 Crime Incident Report, available at 
https://www.bjs.gov/content/pub/pdf/ncvs18_cir.pdf. 

3° Ibid. 








Table 23! 





























Estimates 
Year Total Reported | Not Reported 
2009 287,700| 115,300 172,400 
2010 261,800 90,500 171,400 
2011 255,300 77,000 177,000 
2012 268,000 80,500 175,400 
2013 254,900 88,400 154,300 
2014 266,000} 106,500 146,700 
2015 197,600; 100,300 95,300 
2016 200,100} 104,400 92,100 
2017 197,700; 107,900 86,900 




















Again, we do not see dramatic increases in hate crimes in the NCVS data. The evidence 
does not show that during last decade, a time of expansive growth of electronic communications, 
particularly on the Internet and mobile devices as well as social media, there has been a rise in 
hate crime incidents. 


Hate Crimes in Modern Information and Communication Technologies 


The following section is a review of the literature regarding the commission, advocacy, 
and promotion of hate crimes in a variety of contexts that have developed since the publication 
of the 1993 Report, specifically social media, bots, video games, chat services, and funding 
mechanisms. 


Social Media 


The advent of social media has drastically increased ease of access and reach of 
individuals in their communications. Through early Internet innovations such as email, 





31 Barbara Oudekerk, Hate Crime Statistics Briefing prepared for the Virginia Advisory Committee, U.S. 
Commission on Civil Rights, U.S. Department of Justice (March 29, 2019) available at 
https://www.bjs.gov/content/pub/pdf/hcs1317pp.pdf. 
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discussion boards, and instant messaging systems, to modern social media platforms, the Internet 
facilitates the creation of networks of like-minded persons. Some academics have mused that 
increased communications increases polarization** and even the tendency for violence. ** These 
theories lack any definitive, or even much suggestive, empirical backing. 


No doubt people who commit hate crimes use social media. For instance, Evanna Hu and 
Hugh Brooks describe the way in which the Charlottesville rally organizers used content-specific 
websites, chat rooms, and mainstream social media platforms to plan the event, recruit attendees, 
and distribute propaganda.*4 The authors specifically note that over the course of the rally, 
approximately 35,000 vitriolic messages were circulated on the 44 channels of the 
Charlottesville 2.0 server on the gaming platform Discord. 


But this research, and much like it, fails to demonstrate any causal relationship between 
increased social media use and increased violence. This research does not present even 
comprehensive descriptive data correlating increased hate speech on social media with increased 
hate crimes. 


Similarly, Matthew Williams, et al. write that the far right was quick to realize the value 
of the internet, largely unhindered by law enforcement due to the U.S. s vast constitutional 


protections of speech.*> The authors describe the outcome as the existence of extreme spaces 
that provide a collective virtual identity to previously fragmented and hateful individuals. Again, 
this research—and much like it—lacks empirical support. Further, without examining how the 
internet affects political polarization on all levels, the left and the right, including,*° no 
conclusions can be made about the effect on the internet on political polarization, let alone hate 
crimes. 


2 Lauren Beausoleil, “Free, Hateful, and Posted: Rethinking First Amendment Protection 

of Hate Speech in a Social Media World,” BOSTON COLL. L. REV. 60, no. 7, at 2107 (2019). 

33 Lyrissa Lidsky, “Fifth Annual Criminal Law Symposium: Criminal Law & the First Amendment: the 
First Amendment, the Internet, and the Criminal Law: Incendiary Speech and Social Media,” TEX. TECH 
L. REV. 44, no. 1, at 149-150 (2011). 

4 Evanna Hu and Hugh Brooks, “When Radical Online Political Speech Transitions to Physical 
Violence,” Pacific Standard, (last modified August 24, 2018) https://psmag.com/social-justice/when- 
radical-speech-gets-physical. 

35 Matthew Williams, Pete Burnap, Amir Javed, Han Liu, and Sefa Ozalp, “Hate in the Machine: Anti- 
Black and Anti-Muslim Social Media Posts as Predictors of Offline Racially and Religiously Aggravated 
Crime,” BRITISH JOURNAL OF CRIMINOLOGY 60, no. 1, at 95 (2020) available at 
https://doi.org/10.1093/bjc/azz049., 

6 Brock Simmons, “Seven Antifa Terrorists Arrested On Federal Charges After Weekend Of Carnage In 
Portland,” Gateway Pundit (July 7, 2020) available at 
https://www.thegatewaypundit.com/2020/07/seven-antifa-terrorists-arrested-federal-charges-weekend- 
carnage-portland/; Chuck Goudie, Barb Markoff, Christine Tressel & Ross Weidner, “1,500 rioters 
arrested in Chicago; most not facing serious charges,” ABC7 Chicago (June 3, 2020) 
https://abc7chicago.com/chicago-news-riots-protests-looting/6228227/; Jake Gibson, Michael Ruiz, More 
than 6 dozen alleged rioters face federal charges in weeks of unrest across US,” Fox News (June 24, 
2020) available at https://www.foxnews.com/politics/dozens-alleged-rioters-face-federal-charges-unrest. 
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There is some descriptive data on the prevalence of so-called hate speech,” but the 
inability of the platforms or researchers to arrive at any meaningful, let alone consistent, 
definition of hate speech undermines this research s usefulness for policy purposes. Again, this 


term has no legal meaning or definition under United States law. As the Supreme Court 
repeatedly reminds us, the Constitution in general protects speech that some of us might find 
hateful.” 


The confusion about the meaning of hate speech” is revealed by even the most cursory 


review of the social media platforms policies and practices. While most platforms utilize 


community guidelines that strictly prohibit hate speech, researchers have noted persistent 
inconsistencies relating to the intractable problem in defining hate speech. For example, 
according to Caitlin Carlson, reporting the term "wetback” on Facebook has been found to not 
yield removal, causing confusion over what qualifies as hate speech.** Moreover, Carlson notes 
that community standards do not always match across platforms. While Facebook prohibits 
"organizations and people dedicated to promoting hatred..."°° Twitter only removes specific 


threats and abuse; Twitter s policy is to not remove inflammatory content as long as it does not 


pose direct harm to an individual. Finally, Carlson states that even a cursory search of Instagram 
reveals multiple publicly visible posts containing the N-word and other derogatory terms.*? 


Sharing confusion with the dominant social media platforms, researchers cannot arrive at 
meaningful or consistent definitions of hate speech. A recent review article, though quite 


friendly to the concept of hate speech, recognizes, For the great depth of discussion about the 


harms of it, how it is spread, the appropriate public and private responses to it, and how it should 
be reconciled with theories of free expression, surprisingly little work appears to have been done 


to define the term hate speech itself. Without a clear definition, how will scholars, analysts, and 
regulators know what speech should be targeted?””*! 


For instance, Hawdon, Atte Oksanen, and Pekka R4sanen state that approximately 43% 
of respondents from the United States, the United Kingdom, Germany, and Finland, between the 
ages of 15 and 30, have encountered hate speech online. The number for the United States alone 


has been reported at 53%.*” They define hate speech in the following way: Those expressing 


37 United States v. Schwimmer, 279 U.S. 644 (1929) ( if there is any principle of the Constitution that 


more imperatively calls for attachment than any other, it is the principle of free thought—not free thought 
for those who agree with us but freedom for the thought that we hate.’’). 

38 Caitlin Carlson, ““Censoring Hate Speech in U.S. Social Media Content: Understanding the User’s 
Perspective,” COMM. L. REV. 17, no. 1, at 31 (2017). 

39 Td. 

40 Td. 

“1 Sellars, Andrew F., “Defining Hate Speech,” BERKMAN KLEIN CENTER RESEARCH PUBLICATION NO. 
2016-20 (Dec. 8, 2016). available at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2882244. 

“2 James Hawdon, Atte Oksanen & Pekka Rasdnen, “Exposure to Online Hate in Four Nations: A Cross- 
National Consideration,” DEVIANT BEHAVIOR 38, no. 3, at 260 (2017). 
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hate—be they members of a group or a lone individual—profess attitudes that devalue others 
because of their religion, race, ethnicity, gender, sexual orientation, national origin, or some 
other characteristic that defines a group.”*? 


In contrast, some define hate speech more narrowly. Richard Delgado defines racist hate 
speech more narrowly as_ [l]anguage was addressed to him or her by the defendant that was 


intended to demean through reference to race;” (2) that the plaintiff understood as intended to 
demean through reference to race; and” (3) that a reasonable person would recognize as a racial 


insult.”4 Others use more broad definitions. Calvin Massey states hate speech is any form of 


speech that produces the harms which advocates for suppression ascribe to hate speech: loss of 
self-esteem, economic and social subordination, physical and mental stress, silencing of the 
victim, and effective exclusion from the political arena.”*° 


We hesitate, therefore, to generalize from research that attempts to measure hate speech 
or connect hate speech to hate crimes simply because we have no confidence that this corpus of 
research is measuring the same thing. Lord Kelvin tells us that to measure is to know. 
Conversely, to measure imprecisely is to know little. 


On the other hand, no one doubts that hate criminals use telecommmunications. 
Notorious examples include the live stream of the Christchurch attack on Facebook, which 
stayed live for 17 minutes and was later spread and re-uploaded over 2 million times on various 
other platforms, as well as the Twitch stream of the recent attack on a synagogue in Germany, 
which according to Forbes reporting, was viewed 2,200 times before being completely 
removed.*° 


But, rigorous social scientific empirical research connecting hate speech in social media 
to hate crimes remains scant. In a recent study from the United Kingdom, Williams, et al. have 
claimed to confirm a general temporal and spatial association between online hate speech 
targeting race and religion and offline racially and religiously aggravated crimes independent of 
correlations with offline trigger” events.*’ On the other hand, the data from the 2018 


regression-model study attempting to examine the effect of social media on bias among its users, 





“3 Td. at 166. 

“ Richard Delgado, “Words That Wound: A Tort Action for Racial Insults, Epithets, and Name-Calling,” 
17 HARV. C.R.-C.L. L. REV. 133 (1982). 

“° Calvin R. Massey, “Hate Speech, Cultural Diversity, and the Foundational Paradigms of Free 
Expression,” 40 UCLA L. REV. 103, n.2., at 105 (1992). 

“© Rachel Sandler, “Twitch Says 2,200 Users Viewed Video of Shooting Outside German Synagogue 
Before it was Removed,” Forbes (last modified October 9, 2019) 
https://www.forbes.com/sites/rachelsandler/2019/10/09/twitch-says-2200-users-viewed-video-of- 
shooting-outside-german-synagogue-before-it-was-removed/#34f0c5fb7ca3. 

47 Matthew |. Williams, Pete Burnap, Amir Javed, Han Liu and Sefa Ozalp, “Hate in the Machine: Anti- 
Black and Anti-Muslim Social Media Posts as Predictors of Offline Racially and Religiously 
Aggravated Crime,” THE BRITISH JOURNAL OF CRIMINOLOGY 60, no. 1, at 94 (2020). 
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specifically focusing on religious bias and racist behavior, did not verify a positive correlation.** 
Both studies concluded with the need for further research. There is, however, significant and 
relatively long-studied multidisciplinary bodies of literature on the effects of media consumption 


on individuals. Most of the literature in this area currently rejects the so-called hypodermic 


needle theory” or technological determinism” arguments that ascribe direct relationships 


between media consumption and individual thought or action in favor of theories that take into 
account the contextualization of use and co-constructive nature of communications.*? 


Clay Neff reported that tech leaders have recognized that relying on human teams alone 
to review content will not be enough and that artificial intelligence will have to play a significant 
role.*° That said, there are, of course, significant policy and practical limitations to reliance on 
automated content moderation.*! Interestingly, much of this technology is being developed from 
approaches pioneered by the Chinese Communist Party to stifle political discussion and 
dissent.>? 


Given that all the major social media platforms have rules against hate speech and, in 
fact, employ sophisticated algorithmic artificial intelligence (AI) approaches to enforce these 
often vague and contradictory rules in a manner also used by tyrannous regimes, it is appropriate 
to ask what they gain from it. Certainly, as this Report shows, the platforms have no reasonable 
expectation that their censorship will end hate crimes or even diminish it, as no empirical 
evidence exists linking increased hate speech with hate crimes. 


Further, this censorship poses real dangers to our political system. Under the hate speech 
prohibitions and other censorship rules, the platforms have removed content that many consider 
seriously engaged with pressing political and social issues.*? As with most matters involving 


48 Abdallah Alsaad, Abdallah Taamneh, and Mohamad Noor Al-Jedaiah, “Does Social Media Increase 
Racist Behavior? An Examination of Confirmation Bias Theory,” TECHNOLOGY IN SOCIETY 55, at 43 
(2018). 

” See, e.g., Patti M. Valkenburg, Jochen Peter, & Joseph B. Walther, “Media Effects: Theory and 
Research,” AMSTERDAM SCHOOL OF COMMUNICATION RESEARCH, available at 
https://pdfs.semanticscholar.org/95b4/9d1645 12a0bba6033fc5 1a466de58480a613.pdf?_ga=2.83026582.1 
245738972.1601573190-705737628. 1601573190. 

° Clay Neff, “Call in the Robocops,” THE HUMANIST 78, no. 1, at 20-21 (2018). 

>! “Mixed Messages? The Limits of Automated Social Media Content Analysis,” Center for Democracy 
& Technology (Nov. 2017) available at: https://cdt.org/wp-content/uploads/2017/1 1/Mixed-Messages- 
Paper.pdf. 

°? “WeChat: Bringing CCP Censorship to the Global Stage,” Vision Times (August 27, 2020) available at 
https://visiontimes.com/2020/08/27/wechat-bringing-ccp-censorship-to-the-global-stage.html (Explaining 
that the Chinese app WeChat uses “real-time, automated censorship of images in a chat depending on the 
text that is associated with the pictures.”’); Bill Gertz, “‘Social credit score’: China set to roll out 
‘Orwellian’ mass surveillance tool,” The Washington Times (Dec. 9, 2019) available at 
https://www.washingtontimes.com/news/2019/dec/9/social-credit-system-china-mass-surveillance-tool-/. 
3 Jon Levine, “YouTube censors epidemiologist Knut Wittkowski for opposing lockdown,” New York 
Post (May 16, 2020) available at https://nypost.com/2020/05/16/youtube-censors-epidemiologist-knut- 
wittkowski-for-opposing-lockdown/; Jordan Davidson, “Jack Dorsey Claims Twitter’s Censorship 
Policies Encourage ‘More Speech,” The Federalist (Nov. 17, 2020) available at 
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social media censorship, there is little data beyond anecdote because the social media platforms 
do not disclose data on their censorship practices. Nonetheless, many of the censorship stories 
are quite compelling.** 


At the same time, given that some hate criminals use social media, it would seem 
reasonable to hold the internet platforms accountable that publish these criminals solicitations, 
conspiratorial communications, and efforts at aiding and abetting. But, Section 230 of the 
Communications Decency Act* eliminates liability for platforms republications of speech that 


violates most state criminal law.*° As discussed above, most hate crimes are state laws. 
Revisiting Section 230, either though legislative or regulatory processes, to limit these 
protections would no doubt provide greater protections against hate crimes. 


What is particularly egregious about the Section 230 immunity is its blanket nature. 
Under some influential court decisions, such as Zeran v. Am. Online, Inc., 129 F.3d 327 (4th Cir. 
1997), internet platform have immunity even if they know about the unlawful or even criminal 
content on their websites. Any other entity publishing statements that solicit, conspire to commit, 
or aid and abed hate crimes, would face punishment. But, Section 230 shields social media 
platforms from rules everyone else must follow. Reforming Section 230 would greatly combat 
the problem of soliciting, aiding, and abetting hate crimes online. 


Bots 


Bots, which in this context we are limiting to automated programs used to communicate 
preset messages or messages created through machine learning, can be used for many beneficial 
purposes such as customer relations, streamlining communications between individuals, or even 
art. Bots can also be used to foment political strife, skew online discourse, and manipulate 
marketplaces. 


https://thefederalist.com/2020/1 1/17/jack-dorsey-claims-twitters-censorship-policies-encourage-more- 
speech/. 

4 Jordan Davidson, “Twitter Suspends New York Post Account Over Bombshell Hunter Biden Story,” 
The Federalist (Oct. 13, 2020) available at https://thefederalist.com/2020/10/14/twitter-suspends-new- 
york-post-account-over-bombshell-hunter-biden-story/; Adi Robertson, “Facebook fact-checking is 
becoming a political cudgel,” The Verge (March 3 2020) available at 
https://www.theverge.com/2020/3/3/21163388/facebook-fact-checking-trump-coronavirus-hoax- 
comment-politico-daily-caller; Adam Shaw, “Twitter’s Jack Dorsey says it was ‘mistake’ to censor CBP 
chief over border wall tweet,” Fox News (Nov. 17, 2020) available at 
https://www.foxnews.com/politics/twitter-jack-dorsey-censorship-border-wall-tweet. 

*° 47 U.S.C. § 230. 

°° 47 U.S.C. § 230(e)(3) (“no liability may be imposed under any State or local law that is inconsistent 
with this section”); see also Backpage.com, LLC v. McKenna, 881 F. Supp. 2d 1262, 1275 (W.D. Wash. 
2012) (“If Congress did not want the CDA to apply in state criminal actions, it would have said so”); 
Backpage.com, LLC v. Cooper, 939 F. Supp. 2d 805, 823 (M.D. Tenn. 2013) (“section 39—13-—315 [a state 
criminal law] is likely expressly preempted by CDA section 230(e)(3)); Voicenet Commce’ns, Inc. v. 
Corbett, 2006 WL 2506318, at *4 (E.D.Pa. Aug. 30, 2006) (“[T]he plain language of the CDA provides 
internet service providers immunity from inconsistent state criminal laws.”); see also Michal Lavi, “The 
Good, the Bad, and the Ugly Behavior,” 40 CARDOZO L. REV. 2597, 2683 (2019). 

















15 


Matthew Hines writes that bots are problematic for the same reason that they are useful 
— they amplify the power and efficiency of a single person. Bots can be programmed to 
perform computer-based tasks more quickly and reliably than a human without access to 
technology.°’ Hines also notes that Internet communities are productive environments for 
building consensus, and bots can be used to manipulate that effect. Because it is possible for a 
single independent producer to widely distribute content based on shares and trending tags, bots 
programmed to share specific content can be used "to create a bandwagon effect, to build fake 
social media trends ... and even to suppress opinions of the opposition.”** 


A single bot s increased efficiency relative to a group of human users makes bot use 


lucrative and ripe for abuse. According to Madeline Lamo and Ryan Calo, bots can potentially 
engage in online harassment at an unprecedented scale.*” By automating "trolling," the practice 
of criticizing or threatening certain speakers in response to their views, bots can exacerbate 


highly problematic trends of online hate speech and abuse.® Bots can harass or troll” at scale. 


Jack Balkin also points out that actors could engage in online harassment and threats through the 
use of coordinated armies of trolls consisting of combinations of humans and bots posing as 
humans.°! 


Our research, however, at this time, does not suggest that bots are being systemically 
used to proliferate hate speech or to commit hate crimes.© So far, bot campaigns have been used 
predominantly for commercial and political purposes. Hines describes how this took place during 


the FCC s period for public comments on net neutrality rulemaking, wherein online bots 


contributed hundreds of thousands of comments in support of one particular political position.” 
Bots do, however, possess the potential of being used for offensive speech. For instance, Sara 


Suarez-Gonzalo, studied the example of Tay, Microsoft s chatbox that after launching turned into 
a racist and homophobic robotic hate speaker.™ 


°’ Matthew Hines, “I Smell a Bot: California’s S.B. 1001, Free Speech, and the Future of Bot 
Regulation,” HOUSTON L. REV. 57, no. 2, at 410-411 (2019). 

8 Td. at 411. 

°° Madeline Lamo and Ryan Calo, “Regulating Bot Speech,” UCLA L. REV. 66, no. 4, at 999 (2019). 
oid. 

6! Jack M. Balkin, “Free Speech in the Algorithmic Society: Big Data, Private Governance, and New 
School Speech Regulation,” U.C. DAVIS L. REV. 51, no. 3, at 1191 (2018). 

2 Botnet Road Map Status Update Prepared by the Departments of Commerce and Homeland Security 
(July 28, 2020), available at 

https://www.ntia.doc.gov/files/ntia/blogimages/botnet_road_ map _ status _update.pdf. 

63 Matthew Hines, “I Smell a Bot: California’s S.B. 1001, Free Speech, and the Future of Bot 
Regulation,” HOUSTON L. REV. 57, no. 2, at 410-411 (2019). 

6 Sara Sudrez-Gonazalo, Lluis Mas- Manchon, Frederic Guerrero-Solé, “Tay is you. The attribution of 
responsibility in the algorithmic culture,” OBSERVATION 13, no. 2, at 1-14 (2019). 
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Video Games 


The rise in popularity of videogames has created another new avenue for hate speech. 
Online videogames are social experiences where game cultures and communities can develop. 
Over half of the players who reported experiencing harassment believe it was because of their 
race or gender, and some others, offering impressionistic and anecdotal data, suggests that people 
playing video games are hateful. Again, the significance of these activities and hate crimes is 
obscure, as we have found no causal, or even good descriptive correlative data, showing a 
relationship between video game usage and hate crimes. 


Chat Services 


The Internet allows for much more direct communication among members of hate groups 
and between criminals and victims. This connection has led to new forms of hate crimes through 
messaging and chat services. Citron discusses how hate crimes can be committed through direct 
messages to victims.© Lavi describes how some chat and messaging services are not used to 
communicate hate to victims, but for recruitment, communication, and organization of hate 
groups.°’ Again, the relationship between chat services and hate crimes is obscure, we have 
found no causal, or even good descriptive correlative data, showing a relationship between hate 
crimes and chat services. Indeed, one might expect that limiting chat services would simply 
direct people to older communications technologies such as telephones. 


Funding Mechanisms 


Information and communication technologies can further be used to fundraise in support 
of groups that promote or commit hate crimes. Beach describes terrorists using Paypal, 
GoFundMe, or other online funding platforms to raise money for themselves or for the legal 
defense of those being prosecuted for hate crimes.°* Beach notes that Paypal is quietly cracking 
down on purported white supremacist accounts and describes how the platform is intentionally 
shutting down funding mechanisms for purported white supremacists and other racist hate 
groups. Private banking and financial firms are, of course, free, within certain legal constraints, 
such as the obligation not to discriminate against certain groups such as women, or racial 
minorities, to do business with whomever they chose. On the other hand, accomplice and 
conspirator liability in criminal law already covers financial institutions that knowingly assist or 
plan the performance of hate crimes. 


65 ADL, “Two-Thirds of U.S. Online Gamers Have Experienced Severe Harassment, New ADL Study 
Finds,” (July, 2019) available at: https://www.adl.org/news/press-releases/two-thirds-of-us-online- 
gamers-have-experienced-severe-harassment-new-adl-study; John T. Holden; Thomas A. Baker II., Marc 
Edelman, “The #E-Too Movement: Fighting Back against Sexual Harassment in Electronic Sports,” 
ARIZONA STATE L. J. 52, no. 1, at 1-48 (Spring 2020); Citron, Danielle Keats, “Hate crimes in 
cyberspace,” HARVARD UNIVERSITY PRESS (2014). 

86 Td. 

§7 Michael Lavi, “The Good, the Bad, and the Ugly Behavior,” 40 CARDOZO L. REV. 2597, 2683 (2019). 
°8 Stephanie Beach, “Hashtag Hate: The Need for Regulating Malignant Rhetoric Online,” VT. L. REV. 
44, no. 3, at 141-144 (2019). 
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Online Harassment 


Online blogs, message boards, social media and other websites unfortunately often 
provide the conditions and environment for new modes of harassment, colloquially called online 
harassment. The term encompasses a wide range of behavior which is motivated by bias towards 
another person. There is no consistent definition in law, but it is often defined in state and 
federal law as behavior that is offensive and repetitious. Harassment only in the most extreme 
forms can, under certain statutes, be considered criminal. 


Another new type of harassment that has grown from the Internet is swatting.” 


Elizabeth M. Jaffe defines swatting” as when someone calls 911 and falsely reports a crime or 


emergency. Then a SWAT team is sent to the victim, who usually is live streaming online, to 
investigate the call. The team comes into the house armed and terrifies the victim.” 


Blanch and Hsu outline laws that can be used to combat online harassment.’° The 
stalking provision of the Violence Against Women Act creates a felony offense for any course 


of conduct or series of acts taken by the perpetrator on the Internet that place the victim in 
reasonable fear of death or serious bodily injury, or causes, attempts to cause, or would be 


reasonably expected to cause substantial emotional distress to the victim or the victim s 
immediate family.”’! 18 U.S.C. § 2261A (2015). Further, Section 223 of the Communications 
Act s prohibition against threatening communications can also be used to prosecute harassment. 


This law defines threatening communication as_ [t]he communication in interstate commerce of 


threats to harm a person or property, to kidnap a person, or to damage a person s reputation, is a 


violation of federal law pursuant to 18 U.S.C. § 875.”” Violations of this law are treated as a 
misdemeanor.’> The Supreme Court, in Elonis, ruled that it is not enough under this section to 
show that a reasonable person would have viewed the statements to be threatening, but that the 
perpetrator must possess the sufficient state of mind.“ Even though the statute did not explicitly 
state a mens rea requirement, the Court decided that one must exist.’ The court left open the 
question of whether recklessness would be sufficient to meet the mens rea requirement as it was 
not briefed in the case.” 


°° Elizabeth M. Jaffe, “Swatting: The New Cyberbullying Frontier After Elonis V. United States,” 64 
DRAKE L. REV., at 455, 456 (2016). 

7° Joey L. Blanch; Wesley L. Hsu, “An Introduction to Violent Crime on the Internet,” UNITED STATES 
ATTORNEYS’ BULLETIN 64, no. 3, at 2-12 (May 2016) available at 

https://www Justice. gov/usao/file/85 1856/download 

7 Td. at 5. 

” Td. at 4. 

3 Id. at 8. 

™ Elonis v. United States, 575 U.S. 723 (2015). 

5 Td. 

76 Td. 
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Constitutionally Protected Speech and other Protections 


The 1993 Report detailed the crucial role that the First Amendment plays in this area, 
given that telecommunications involves speech and expression.’’ The First Amendment serves 
as an important limitation to government action on private speech and expression, and thus 
“furnishes a critical starting point for discussion of potential government responses to the use of 
telecommunications to advocate and encourage hate crimes.””® 


While First Amendment protections are generally broad, the First Amendment does not bar 
government action on hate crimes, the central subject of the Report and this update. But, the 
First Amendment does bar all efforts for the government to regulate hate speech. The Supreme 
Court could not be clearer: “Speech that demeans on the basis of race, ethnicity, gender, religion, 
age, disability, or any other similar ground is hateful; but the proudest boast of our free speech 
jurisprudence is that we protect the freedom to express “the thought that we hate.” 


The most important evolution in communications technology since the 1993 Report has 
been the rise of the commercial Internet. While the 1993 Report highlighted the relationship 
between the First Amendment and some of the latest networked technology at the time (e.g., 
Bulletin Board Systems), the Report did not discuss the Internet. Therefore, this update provides 
the following observations on First Amendment limitations to government action on speech on 
the Internet, including speech which may advocate or encourage hate crimes.. These 
observations can serve as a starting point to conceptualize possible government approaches, but 
they are not intended to be all-inclusive or to evaluate the merits of particular laws or current 
legislative efforts. 


The Supreme Court has explicitly considered — and rejected — the argument that under the 
First Amendment, government content-based regulation of speech on the Internet should be 
evaluated under the more deferential standard given to government regulation of speech on 
broadcast television and radio. 

In Reno v. ACLU, the Supreme Court struck down on First Amendment grounds a 
federal law that criminalized indecent transmissions via telecommunications done with 
knowledge that the recipient would be a minor (the indecent transmissions provision), as well as 
the knowingly sending or displaying of depictions of sexual or excretory activities or organs in a 
manner that would be available to minors and in a way that would be patently offensive to 
contemporary community standards (the patently offensive display provision).*° Congress had 
passed these provisions as part of the Communications Decency Act, with the explicit intent of 


7” U.S. Dept. of Commerce, Nat’! Telecomm. and Info. Admin., The Role of Telecommunications in Hate 
Crimes (1993) (Although analyzing the constitutionality of laws and regulations related to hate crimes 
and hate speech can sweep in other constitutional provisions, including the Fifth, Thirteenth, and 
Fourteenth Amendments, this Update focuses on the First Amendment because of its particular impact on 
speech on the Internet.). 

78 Id. at 32. 

” United States v. Schwimmer, 279 U.S. 644 (1929) (“if there is any principle of the Constitution that 
more imperatively calls for attachment than any other, it is the principle of free thought—not free thought 
for those who agree with us but freedom for the thought that we hate.’’). 

8° Reno v. Am. Civil Liberties Union, 521 U.S. 844, 859-60; 883 (1997) (The Supreme Court did uphold 
as constitutional the part of the law that prohibited obscene transmissions). 
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protecting minors online.*! The Court held these provisions unconstitutional as a violation of the 
First Amendment.*” Specifically, the Court noted that the statute “unquestionably silence some 
speakers whose messages would be entitled to constitutional protection.”®? The Court also 
concluded that the provisions would curtail a significant amount of protected speech between 
adults; *4 the provisions defined neither “indecent” nor “patently offensive,” thus including “non- 
pornographic material with serious educational or other value.”*> The Court also rejected the 
provisions’ reliance on contemporary community standards to define regulated speech, noting 
that “the community standards criterion as applied to the Internet means that any communication 
available to a nationwide audience will be judged by the standards of the community most likely 
to be offended by the message.”*° 


In evaluating the constitutionality of the law, the Court contended for the first time with 
government regulation of speech on the Internet. The Court recognized that there is a 
government interest “in protecting the physical and psychological well-being of minors. 
However, the Court determined that speech on the Internet was not subject to the lesser degree of 
protection from government regulation that the Court in earlier cases had determined was 
appropriate in regulating broadcast television and radio.** The Court cited the extensive history 
of broadcast regulation,*? the scarcity of available frequencies,”’ and the invasive nature of 
broadcast?! as reasons for differentiation. Instead, a government content-based restriction on 
speech online has to survive strict scrutiny, which means that the government has to demonstrate 
that a law or regulation is narrowly tailored to achieve a compelling government interest and 
uses the least restrictive means to achieve the purpose.”” Thus, as the Court would later note 


9987 





81d. at 858. 

821d. at 866-870 (1997) (The Supreme Court did not decide whether the statute was so vague as to also 
violate the Fifth Amendment, but the Court noted that “the many ambiguities concerning the scope of [the 
statute’s] coverage render it problematic for purposes of the First Amendment.”). 

83 Id. at 874. 

84 Id. at 874-79. 

85 Td. at 877. 

86 Td. at 877-78 but see Ashcroft v. ACLU, 535 U.S. 564, 585 (2002) (A statute’s “reliance on community 
standards to identify material that is harmful to minors does not by itself render the statute substantially 
overbroad for purposes of the First Amendment.”). 

87Reno, 521 U.S. at 869 (quoting Sable Comm. of CA, Inc. v. FCC 492 U.S. 115, 126 (1989)). 

88 Td. at 870 (The Court noted that “our cases provide no basis for qualifying the level of First Amendment 
scrutiny that should be applied to [the Internet.” ]); see also FCC v. Pacifica Found., 438 U.S. 726 (1978). 
89 Reno, 521 U.S. at 870 (citing Red Lion Broad. Co. v. FCC, 395 U.S. 367, 398 (1969)). 

%° Td. (citing Turner Broad. Sys. Inc. v. FCC, 512 U.S. 622, 637-638 (1994)). 

1 Td. (citing Sable Comm. of CA, Inc. v. FCC, 492 U.S. 115, 126 (1989)). 

% Td. at 874; see also PSINet, Inc. v. Chapman, 362 F.3d 227, 233 (4th Cir. 2004) (noting that the 
Supreme Court in Reno applied strict scrutiny to regulation of Internet speech); ACLU v. Johnson, 194 
F.3d 1149, 1156 (10th Cir. 1999) (same); see also U.S. v. Playboy Ent. Grp., Inc., 529 U.S. 803, 813 
(2000) (describing the strict scrutiny standard) (Thus, even when a law or regulation specifically targets 
unprotected speech or expression, a court may find a First Amendment violation when the actions taken 
to comply with the law or regulation suppress a significant amount of protected speech.); see, e.g., U.S. v. 
Playboy Ent. Grp., Inc., 529 U.S. 803 (2000) (This is not suggest that a statute that criminalizes any form 
speech or expression on the Internet would violate the First Amendment.); see, e.g., U.S. v. Dinghra, 371 
F.3d 557 (9th Cir. 2004) (Moreover, not all speech regulation per se is subject to strict scrutiny; an 
important exception to a private entity’s exemption from First Amendment restriction is commercial 
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regarding Reno, “the mere possibility that user-based Internet screening software would soon be 
widely available was relevant to [the Court’s] rejection of an overbroad restriction of indecent 
cyber speech.””? 


A speaker’s decision to remain anonymous is protected under the First Amendment. 


Anonymity is an important aspect of the online environment, and anonymity as a way to 
protect the speaker’s identity can be used for legitimate or illegitimate purposes. In McIntyre v. 
Ohio Elections Com’n, the Supreme Court noted that “an author’s decision to remain 
anonymous, like other decisions concerning omissions or additions to the content of a 
publication, is an aspect of freedom of speech and protected by the First Amendment” and that 
“t]he freedom to publish anonymously extends beyond the literary realm.”°* Anonymity, the 
Court reasoned, “exemplifies the purpose behind the Bill of Rights and of the First Amendment 
in particular: to protect unpopular individuals from retaliation—and their ideas from 
suppression—at the hand of an intolerant society.””> While the right to remain anonymous is 
not absolute,”° in the context of online speech, the Sixth Circuit has determined that even a 
plaintiff who prevails in a lawsuit is not automatically entitled to unmask the defendant’s identity 
to the plaintiff, although there is a presumption in favor of unmasking.”’ The Sixth Circuit noted 
that “courts must consider both the public interest in open records and the plaintiff's need to 
learn the anonymous defendant’s identity in order to enforce its remedy. . .[fJurther, where a Doe 
defendant’s speech is found to be beyond the protection of the First Amendment, countering the 
presumption will require a showing that the Doe defendant participates in a significant amount of 
other, non-infringing anonymous speech that would be chilled if his identity were revealed.””® 


speech, which historically has held the government to lower level of judicial scrutiny.); see generally 
Valarie Brannon, “Assessing Commercial Disclosure Requirements Under the First Amendment,” Cong. 
Research Serv., R45700 (2019). 

3 U.S. y. Playboy Ent. Grp., Inc., 529 U.S. 803, 814 (2000) (referencing Reno); see also Ashcroft, 542 
U.S. 656, 670 (2004) ( Absent a showing that the proposed less restrictive alternative would not be as 


effective, [. . .] the more restrictive option preferred by Congress could not survive strict scrutiny.”’) 
(internal citations omitted). 

%4 McIntyre v. Ohio Elections Com’n, 514 U.S. 334, 342 (1995). 

dat 301. 

% See, e.g., In re Anonymous Online Speakers, 661 F.3d 1168 (9th Cir. 2011) (“[N]ature of the speech 
should be a driving force in choosing a standard by which to balance the rights of anonymous speakers in 
discovery disputes.”) (citing Perry v. Schwarzenegger, 591 F.3d 1147, 1160-61 (9th Cir. 2010); Doe v. 
Reed, 561 U.S. 186 (2010)). 

7 Signature Mgmt. Team, LLC v. Doe, 876 F.3d 831 (6th Cir. 2017) (In earlier proceedings, a district 
court had granted plaintiff's motion for summary judgment to hold defendant liable for copyright 
infringement of plaintiff’s work by posting the work on defendant’s blog, and the district court had 
ordered that the defendant destroy all copies in the defendant’s possession.); id. at 835 (In determining 
whether the plaintiff could unmask the defendant’s identity at the discovery stage, the district court 
applied a test designed “to balance the magnitude of the harms that would be caused to the competing 
interests by a ruling in favor of plaintiff and by a ruling in favor of defendant.”); id. at 834 (internal 
quotation marks and citations omitted); see also Signature Mgmt. Team, LLC v. Doe, No. 13-cv-14005, 
2015 WL 13036681, *5 (E.D. Mich. 2015) (providing a longer discussion about balancing tests that 
courts have applied pre-judgment and during the discovery process). 

% Signature Mgmt. Team, LLC v. Doe, 876 F.3d 831, 837 (6th Cir.2017). 
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Government generally may not restrict access to online services in a manner so broad as to 
prevent users from exercising their First Amendment rights. 


The Supreme Court in Packingham v. North Carolina held unconstitutional a statute that 
made it a felony for registered sex offenders to access commercial social networking sites, where 
the sex offender knows that the sites permit minors to be members.”? The Court held that the 
statute was an impermissible restriction on lawful speech — even assuming that the statute was 
content neutral and thus subject to lesser (intermediate) scrutiny. '°’ The Court concluded that the 
statute “burden[ed] substantially more speech than [was] necessary to further the government’s 
legitimate interests”!°! and noted that “to foreclose access to social media altogether is to prevent 
the user from engaging in the legitimate exercise of First Amendment rights.”!°? The Court also 
stated that “this opinion should not be interpreted as barring a State from enacting more specific 
laws than the one at issue. [. . .] [I]t can be assumed that the First Amendment permits a State to 
enact specific, narrowly tailored laws that prohibit a sex offender from engaging in conduct that 
often presages a sexual crime, like contacting a minor or using a website to gather information 
about a minor.”!% In the context of online service access restrictions as part of a judicial 
sentence, courts have differentiated from Packingham by noting the Supreme Court there 
“invalidated only a post-custodial restriction and expressed concern that the statute applied even 
to persons who have already served their sentence.” !™ 


Government regulation of a private entity may transform that private entity into a state actor or 
public forum subject to the First Amendment. 


Ordinarily, the First Amendment does not apply to private actors.!°° However, the state- 
action doctrine provides that a private entity can be held to the standard of a state actor when the 
private entity exercises a function traditionally exclusively reserved to the state.”!°° In 
Manhattan Community Access Corporation v. Halleck, the Supreme Court determined that a 
private entity s operation of public access channels on a cable system” did not transform the 
entity into a state actor, and therefore, was not subject to the constraints of the First Amendment, 


despite New York State s extensive regulation of the entity. !°” 





°° Packingham v. North Carolina, 137 S.Ct. 1730, 1737 (2017). 

100 Tq. at 1736. 

101 Td. 

102 Td. at 1737. 

103 Td. 

4 U.S. v. Carson, 924 F.3d 467, 473 (8th Cir. 2019) (italics in original) (The Eight Circuit also highlights 
decisions from other courts of appeal that “have rejected a similar argument [as that in Packingham] in 
challenges to supervised released conditions forbidding access to the Internet.”’); id. (citing cases) (Most 
of these cases involved a review for plain error of the respective lower courts’ decisions.). 

105 Manhattan Cmty. Access Corp. v. Halleck, 139 S.Ct. 1921, 1926 (2019). 

106 Td, (citing Jackson v. Metropolitan Edison Co., 419 U.S. 345, 352 (1974)) (Gnternal quotation marks 
omitted). 

107 Td. at 1931-32. 
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On the other hand, courts have also ruled that social media can be considered public 
forum subject to the First Amendment. Knight First Amendment Inst. at Columbia Univ. v. 


Trump tuled that President Trump s Twitter account was a public forum to which the First 


Amendment applies. The court ruled that the President and Scavino [the White House social 


media director] exert governmental control over certain aspects of the @realDonaldTrump 
account, including the interactive space of the tweets sent from the account. That interactive 


space is susceptible to analysis under the Supreme Court s forum doctrines, and is properly 
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characterized as a designated public forum.”!°° Similarly, Robinson v. Hunt County’ involved 


a governmental social media account, in particular, a sheriff office s Facebook page. The court 
ruled that public fora First Amendment rules applied. 


Recommendations 


Pursuant to Congress’s request, and in line with the findings of this report, the following are 
a series of recommendations as to how the U.S. Government could better combat the use of 
electronic communications technologies in advocating and encouraging hate crimes. 


e Respect the First Amendment 


As the Supreme Court has recognized, the social media platforms have become our 
Nation’s public square, and their power to track users and gather and analyze their data 
exceeds that of the most enthusiastic small town gossip.'!° Too close government 
collaboration, even in the worthy goal of fighting crime, particularly hate crime, may 
result in government censorship via corporate proxy.!'! Overly compliant private 
internet firms will marry De Tocqueville’s nightmare of an unbridled democratic tyranny 
desirous of stamping out nonconformity and minority views to Orwell’s dystopia of an 
all-seeing state. With that caveat, we make the following, additional recommendations. 


e Strengthen Collaboration Among State, Local And Federal Officials 


It is critical that local and state law enforcement work together because of 
multijurisdictional nature of online hate crimes. The problems that occur online may 
require one jurisdiction to investigate and another to prosecute with the assistance of the 
Federal Government which has unique resources and experience to lend to the problem. 





108 Knight First Amendment Inst. at Columbia Univ. v. Trump, 302 F.Supp.3d 541, 580 (2018). 

109 921 F.3d 440, 447-448 (Sth Cir. 2019) (“Because Robinson alleges viewpoint discrimination, it is 
immaterial whether the Facebook page is analyzed as a limited or designated public forum. The First 
Amendment ‘forbid[s] the State to exercise viewpoint discrimination’ in either setting, 

even when the limited public forum is one of its own creation,” guoting Rosenberger v. Rector and 
Visitors of Univ. of Va., 515 U.S. 819, 829 (1995)). 

10 Packingham v. North Carolina, 137 S. Ct. 1730, 1737 (2017). 

111 See Seth F. Kreimer, “Censorship by Proxy: The First Amendment, Internet Intermediaries, and the 
Problem of the Weakest Link,” 155 U. PA. L. REv. 11, 14 (2006). 
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e Develop Technologies to Help Users Screen and Control Content 


The U.S. Government should continue to “encourage the development of technologies 
which maximize user control over what information is received by individuals, families, 
and schools who use the Internet and other interactive computer services.” !! 
Empowering users to take charge of their online experiences by giving them the tools to 
control what they see and with whom they interact could help foster a healthier digital 
ecosystem. More research is needed into how technology can be leveraged to best 
facilitate a greater degree of user control in today’s Internet, and whether and how any 
proposed models promote speech online. !" 


e Improve platform transparency and disclosure 


As NTIA’s recent petition to the Federal Communications Commission (FCC) argued,!'* 
very little is publicly known about social media content monitoring and usage. Public 
disclosure of content moderation practices would allow policymakers—and citizens—to 
better understand the role electronic media play in hate crimes. 


Conclusion 


The preceding report provided an update to the findings of the 1993 Report with respect 
to developments in information and communications technologies. This update was, of 
necessity, limited in scope relative to the 1993 Report due in large part to the explosive 
expansion of these technologies, their uses, and their centrality to many aspects of modern 
society, both noble and odious. But, our conclusions remain the same—and are consistent with 
leading scholars and researchers of free speech. We found no evidence linking electronic 
communications to hate crimes. As Erwin Chemerinsky states, there is “no reason to believe that 
censoring hate speech will make hate crimes less likely.”!'> Rather, regulating hate speech on 


'l2 47 U.S.C. § 230(b)(3); see also 47 U.S.C. § 230(c)(2)(B); 47 U.S.C. § 230(d). 

113 National Telecommunications and Information Administration, Petition for Rulemaking, Docket No. 
RM- 11862 at 27-31 (Filed July 27, 2020); Mike Masnick, “Protocols, Not Platforms: A Technological 
Approach to Free Speech,” KNIGHT FIRST AMENDMENT INSTITUTE AT COLUMBIA UNIVERSITY (For 
example, one commentator has recommended the creation of “open protocols” in for online 
communications, arguing that such a shift “would allow end users to determine their own tolerances for 
different types of speech but make it much easier for most people to avoid the most problematic speech, 
without silencing anyone entirely or having the platforms themselves make the decisions about who is 
allowed to speak.”’) available at https://knightcolumbia.org/content/protocols-not-platforms-a- 
technological-approach-to-free-speech. 

™4 National Telecommunications and Information Administration, Petition for Rulemaking, Docket No. 
RM- 11862 at 27-31 (Filed July 27, 2020). 

115 “Hate speech is infecting America, but trying to ban it is not the answer,” Chicago Tribune (Oct. 31, 
2018), available at https://www.chicagotribune.com/opinion/commentary/ct-perspec-hate-speech-censor- 
first-amendment-1101-20181031-story.html. 














24 


electronic communications platforms, as experts such as Nadine Strossen have concluded, is “at 
best ineffective and at worst counterproductive” in combating hate crimes. '!° 


On the other hand, electronic media, particularly the large social media platforms, 
currently used to publish criminal threats, solicitations, and other criminal speech that aides and 
abets hate crimes. In contrast “hate speech,” this speech is illegal. Nonetheless the companies 
who control that media have immunity pursuant to 47 U.S.C. § 230(e)(3) and have no liability 
for hosting this criminal speech. To the degree Congress wishes to counteract the role of 
electronic media in the encouragement of hate crimes, revisiting these immunities might be 
considered. As mentioned above, the Department of Justice’s recent proposals for revising 
section 230 offer an excellent guide for this much needed reform. !” 


16 T aura E. Adkins, “Former ACLU president says censoring hate speech can backfire — just like it did in 
Nazi Germany,” Sept. 3, 2020, available at https://www.jta.org/2020/09/03/opinion/former-aclu- 
president-says-censoring-hate-speech-can-backfire-just-like-it-did-in-nazi-germany. 
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